Data

This data was taken from a clean dataset that Jae emailed out on October 25th, 2019.

Jae organized the retina by six different methods. He created OCTA variable scores for the overall image, 2 hemifields, 4 quadrants, 6 sectors, 12 clock hours, and an annular ring.

For now, only the overall image scoring will be analyzed.

I manually removed underscores from the names of each patient because it was causing uneven splitting of variables. This modified dataset is saved as all_quantification_10_24_19_DG.xlsx. I updated the names for every tab of the file.

Data Dictionary

variable label
ptid Patient Identification
eye_imaged Eye Imaged
time_stamp Exam Date and Time
name_last Last Name
name_first First Name
female Female Sex
image_area Image Area
image_method Image Method
image_size Image Size
v_diameter Vessel Diameter (units?)
v_area_density Vessel Area Density (units?)
v_skeleton_density Vessel Skeleton Density (units?)
v_perimeter_index Vessel Perimeter Index (units?)
v_complexity_index Vessel Complexity Index (units?)
flow_impair_zone Flow Impairment Zone (units?)
flux Flux (units?)
msrid Measurement ID

Data Wrangling

Rename Variables

This section will rename the variables to match the dictionary

Format Variables

This section will format variables appropriately.

Label Variables

This section will label variables appropriately.

Subsetting Cohorts

Intra-Visit Data

Intra-visit data was restricted to all measurements of the same eye that occurred on the same day. Some people had multiple measurements on multiple days. Their first day was included only.

Inter-Visit Data

Inter-visit data was restricted to the first measurement of the day for each eye. If a patient had their right eye measured:

  • 3 times on Monday

  • 5 times on Wednesday, and

  • 2 times on Friday,

Then they would have three observations in this dataset:

  • The first measurement from Monday,

  • The first measurement from Wednesday, and

  • The first measurement from Friday.

Data Flow Chart

This tallies how many participants, how many eyes, and how many measurements are included in the two datasets.

dataset n_participants n_eyes n_measurements
Intra-Visit 84 128 320
Inter-Visit 84 129 353

Single Eye Participation

Venugopal (2019) et al. ran the analysis including all eyes.1 Then they re-ran the analysis after selecting only a single eye randomly from each individual. They conclude minimal difference in the findings, and they report only the analysis that includes all eyes:

“The entire analysis was repeated considering one eye per subject (one eye was randomly chosen from subjects contributing both eyes for the primary analysis) and all the results were similar to the primary analysis.”1

Based on this, analysis was performed on all eyes. If time permits, remove a random eye later.

Table 1: Summary Statistics

The Venugopal papers dichotomize glaucoma. But there’s no glaucoma indicator variable in this data.

Intra-Visit OCTA Summary Statistics

Variables in intra-visit dataset

Overall
n 320
Sociodemographics
Female Sex = Male (%) 160 ( 50.0)
Image Specifications
Eye Imaged = OD (%) 165 ( 51.6)
Image Area = ONH (%) 320 (100.0)
Image Method = Angiography (%) 320 (100.0)
Image Size = 6x6 (%) 320 (100.0)
OCTA Measures
v_diameter (mean (SD)) 26.67 (1.49)
v_area_density (mean (SD)) 0.32 (0.07)
v_skeleton_density (mean (SD)) 0.12 (0.03)
v_perimeter_index (mean (SD)) 0.28 (0.06)
v_complexity_index (mean (SD)) 6853.47 (1636.56)
flow_impair_zone (mean (SD)) 12.95 (5.53)
flux (mean (SD)) 0.09 (0.03)

Inter-Visit OCTA Summary Statistics

Variables in inter-visit dataset

Overall
n 353
Sociodemographics
Female Sex = Male (%) 171 ( 48.4)
Image Specifications
Eye Imaged = OD (%) 176 ( 49.9)
Image Area = ONH (%) 353 (100.0)
Image Method = Angiography (%) 353 (100.0)
Image Size = 6x6 (%) 353 (100.0)
OCTA Measures
v_diameter (mean (SD)) 26.88 (1.57)
v_area_density (mean (SD)) 0.31 (0.07)
v_skeleton_density (mean (SD)) 0.12 (0.03)
v_perimeter_index (mean (SD)) 0.27 (0.06)
v_complexity_index (mean (SD)) 6596.93 (1637.05)
flow_impair_zone (mean (SD)) 13.77 (5.63)
flux (mean (SD)) 0.09 (0.03)

Exploratory Data Analysis

Scatter Plots

Scatter plots of OCTA variables by Subject ID for about 10 individuals randomly sampled. These are separated by intra- and inter-visit datasets, and are useful for understanding the data better.

Vessel Area Density

Vessel Diameter

Vessel Skeleton Density

Vessel Perimeter Index

Vessel Complexity Index

Flow Impairment Zone

Flux

Spaghetti Plots

Spaghetti plots of OCTA variables for about 10 individuals randomly sampled.

Vessel Area Density

Vessel Diameter

Vessel Skeleton Density

Vessel Perimeter Index

Vessel Complexity Index

Flow Impairment Zone

Flux

Bland-Altman Plots

Coding of Bland-Altman (BA) plots is explained in this blog post.

Intra-Visit Bland-Altman Plots

Intra-Visit Bland-Altman Plots of OCTA Variables

Vessel Area Density

Vessel Diameter

Vessel Skeleton Density

Vessel Perimeter Index

Vessel Complexity Index

Flow Impairment Zone

Flux

Outliers

Assess some of the observations outside of expected limits.

msrid ptid eye_imaged time_stamp exam_date image_area image_method image_size v_diameter v_area_density v_skeleton_density v_perimeter_index v_complexity_index flow_impair_zone flux female n_msr_day order_obs
50 142119 OS 2017-11-17 08:37:58 2017-11-17 ONH Angiography 6x6 27.17439 0.2554167 0.09399167 0.2203111 5443.976 17.8695 0.06434819 Male 2 1
51 142119 OS 2017-11-17 08:38:33 2017-11-17 ONH Angiography 6x6 26.59994 0.2882917 0.10838056 0.2482833 6125.699 12.9055 0.07575618 Male 2 2
55 142119 OD 2017-11-17 08:35:26 2017-11-17 ONH Angiography 6x6 27.43941 0.2346222 0.08550556 0.2009250 4929.370 20.4151 0.05710792 Male 2 1
56 142119 OD 2017-11-17 08:35:58 2017-11-17 ONH Angiography 6x6 27.39983 0.2725750 0.09948056 0.2323556 5674.306 18.4770 0.07224138 Male 2 2
121 1787031 OD 2019-04-10 11:47:25 2019-04-10 ONH Angiography 6x6 26.79852 0.2269611 0.08469167 0.1965333 4875.438 19.4988 0.06092570 Female 2 1
122 1787031 OD 2019-04-10 11:47:52 2019-04-10 ONH Angiography 6x6 26.37865 0.3541333 0.13425000 0.3041139 7481.666 10.8069 0.10075818 Female 2 2
139 1921477 OD 2019-02-27 11:28:13 2019-02-27 ONH Angiography 6x6 26.62653 0.2598083 0.09757500 0.2272306 5693.417 17.8298 0.07133596 Female 3 1
140 1921477 OD 2019-02-27 11:28:36 2019-02-27 ONH Angiography 6x6 27.63048 0.2244056 0.08121667 0.1943750 4823.251 21.3663 0.05862028 Female 3 2
175 2023329 OD 2018-08-08 11:12:24 2018-08-08 ONH Angiography 6x6 29.50645 0.2007750 0.06804444 0.1675528 4005.772 20.6183 0.05156483 Male 3 1
176 2023329 OD 2018-08-08 11:12:58 2018-08-08 ONH Angiography 6x6 30.15018 0.1795694 0.05955833 0.1496167 3571.249 25.0240 0.04347086 Male 3 2
592 7416893 OS 2018-08-24 09:50:27 2018-08-24 ONH Angiography 6x6 27.42530 0.2807056 0.10235278 0.2405694 5906.396 12.6800 0.07410612 Female 3 1
593 7416893 OS 2018-08-24 09:51:00 2018-08-24 ONH Angiography 6x6 26.37793 0.3191583 0.12099444 0.2749278 6784.591 9.4247 0.08662689 Female 3 2

Inter-Visit Bland-Altman Plots

Inter-Visit Bland-Altman Plots of OCTA Variables

Vessel Area Density

Vessel Diameter

Vessel Skeleton Density

Vessel Perimeter Index

Vessel Complexity Index

Flow Impairment Zone

Flux

Outliers

Assess some of the observations outside of expected limits.

msrid ptid eye_imaged time_stamp exam_date image_area image_method image_size v_diameter v_area_density v_skeleton_density v_perimeter_index v_complexity_index flow_impair_zone flux female order_obs
42 1278865 OD 2017-08-28 15:32:52 2017-08-28 ONH Angiography 6x6 24.43285 0.4538944 0.18577222 0.3991667 10056.487 4.1252 0.15370054 Female 1
138 1921477 OD 2017-02-10 16:07:17 2017-02-10 ONH Angiography 6x6 27.62038 0.2523889 0.09137778 0.2192500 5456.342 18.7283 0.06879399 Female 1
187 2083485 OD 2017-01-20 15:57:02 2017-01-20 ONH Angiography 6x6 27.71777 0.2504917 0.09037222 0.2164278 5357.046 19.4155 0.06793875 Male 1
188 2083485 OD 2018-01-19 15:20:38 2018-01-19 ONH Angiography 6x6 26.43215 0.2932500 0.11094444 0.2540056 6302.909 15.5004 0.08255908 Male 2
272 455685 OS 2018-08-24 08:36:16 2018-08-24 ONH Angiography 6x6 26.48730 0.3183111 0.12017500 0.2757667 6844.227 13.7914 0.08894135 Female 2
295 6743869 OS 2018-07-20 08:24:06 2018-07-20 ONH Angiography 6x6 27.92273 0.2485278 0.08900556 0.2162194 5388.988 17.5764 0.06469750 Male 2
330 6862281 OD 2016-09-02 17:02:28 2016-09-02 ONH Angiography 6x6 27.30537 0.2057611 0.07535556 0.1757750 4301.734 21.2893 0.05488608 Female 1
331 6862281 OD 2017-03-17 13:34:14 2017-03-17 ONH Angiography 6x6 26.37865 0.3541333 0.13425000 0.3041139 7481.666 10.8069 0.10075818 Female 2
535 7262991 OD 2017-08-10 10:31:02 2017-08-10 ONH Angiography 6x6 25.84683 0.3080583 0.11918611 0.2701472 6786.732 14.3854 0.08608585 Male 1

Intra-Visit Repeatability

Intra-Visit Repeatability of OCTA Variables

Calculate the within-subject standard deviation \((S_w)\), the within-subject coefficient of repeatability \((CR_w)\), and the within-subject coefficient of variation \((CV_w)\) for a given measurement variable \(x\).

Table 2: Repeatability

Table 2 shows repeatability estimates of vessel density measurements.

This table was modeled after Table 2 in Venugopal (2018).2 They stratified these values by dichotomous glaucoma status, but we’re not sure if that is appropriate.

octa_var µ Sw CRw CVw
Vessel Diameter (units?) 26.712 0.285 0.791 1.069
Vessel Area Density (units?) 0.315 0.014 0.039 4.474
Vessel Skeleton Density (units?) 0.119 0.006 0.017 4.997
Vessel Perimeter Index (units?) 0.273 0.012 0.034 4.463
Vessel Complexity Index (units?) 6793.597 304.110 842.951 4.476
Flow Impairment Zone (units?) 13.192 1.501 4.161 11.380
Flux (units?) 0.092 0.005 0.014 5.407

I still need to figure out how they calculated the 95% CIs, which I think is just \(1.96*\sqrt{SE}\) because they all appear symmetric, but I’m not sure yet. Then I need to stratify these stats by glaucoma diagnosis (yes/no). - DG 8/29/2019

Bruce showed me a paper that has the calculation of the confidence intervals. It’s not difficult to manually code them, just need to read the paper. - DG 9/4/2019

Calculating Repeatability Statistics

Explanations of repeatability statistic calculations

Within-Subject Mean \((\mu_w)\)

\(\mu_w\) is the average measurement of both eyes for each individual. The overall mean \((\mu)\) is calculated by taking the mean of \(\mu_w\) across the dataset. The overall mean is included in the final table for reference.

\[ \mu_w = \frac{1}{M}\sum_{i=1}^{M} x_i \]

\[\text{where } M \text{ is the number of measurements per eye,}\]

Within-Subject Standard Deviation \((S_w)\)

Find \(S_w\) by first calculating the variance of the measurements per individual eye. The equation below is general for any number of measurements, but in this study there are only 2 measurements per eye.

\[ \sigma^2_{measurement} = \frac{1}{M}\sum_{i=1}^{M} (x_i - \mu_w)^2 \]

\[ \text{where } M \text{ is the number of measurements per eye} \]

To calculate \(S_w\), average of the variance of measurements \((\sigma^2_{measurement})\) for all eyes measured, then take the square root.

\[ Sw = \sqrt{\frac{1}{N}\sum_{i=1}^{N}\sigma^2_{measurement,i}} \]

\[ \text{where } N \text{ is the number of eyes measured} \]

Within-Subject Coefficient of Repeatability \((CR_w)\)

The \(CR_w\) provides the uncertainty of repeated measures.

\[ CRw = \sqrt2 * 1.96 * Sw \]

The \(CR_w\) can be interpreted as:

“The difference between two measurements for the same subject is expected to be less than [\(CR_w\)] for 95% of pairs of observations.”3

Within-Subject Coefficient of Variation \((CV_w)\)

\[ CVw = 100 *\frac{Sw}{\mu} \]

Intraclass Correlation Coefficient

Intraclass Correlation Coefficient (ICC)

Intra-Visit ICC

V1 ICC LowerCI UpperCI N k varw vara
v_diameter 0.755 0.679 0.821 84 3.802 0.545 1.679
v_area_density 0.817 0.756 0.868 84 3.802 0.001 0.004
v_skeleton_density 0.817 0.756 0.868 84 3.802 0.000 0.001
v_perimeter_index 0.814 0.752 0.866 84 3.802 0.001 0.003
v_complexity_index 0.810 0.747 0.863 84 3.802 513513.102 2188411.751
flow_impair_zone 0.800 0.734 0.855 84 3.802 6.192 24.709
flux 0.822 0.762 0.872 84 3.802 0.000 0.001

Inter-Visit ICC

V1 ICC LowerCI UpperCI N k varw vara
v_diameter 0.684 0.597 0.763 84 4.188 0.789 1.705
v_area_density 0.768 0.697 0.830 84 4.188 0.001 0.004
v_skeleton_density 0.772 0.703 0.833 84 4.188 0.000 0.001
v_perimeter_index 0.767 0.696 0.829 84 4.188 0.001 0.003
v_complexity_index 0.766 0.695 0.829 84 4.188 632550.281 2073334.366
flow_impair_zone 0.720 0.639 0.792 84 4.188 8.956 22.985
flux 0.760 0.688 0.824 84 4.188 0.000 0.001

Calculating ICC Statistic

\[ \text{ICC} = \frac{\text{between}}{\text{between} + \text{within}}\]

“In the output for the random effects model ‘SID’ is the estimated between-group variance, which in the ICCest output is called ‘vara’ (variance among groups). The within-group variance is ‘Residual’ in the random effects model output, while in ICCest it’s called ‘varw.’ You can see that the values for those two variances match perfectly in the two different outputs. The ICC is defined as between/(between+within). That calculation is not part of the random effects model output, but it is part of the ICCest output, along with confidence intervals for the ICC estimate. The ICC are really high for intravisit (around 97%) and a little lower for intervisit (about 94%). If we were to calculate these ICC for glaucoma and controls separately I think they would be significantly lower because of the reduced between-group variability.” - BB 9/4/2019

Mixed Effects Regression Models

Model Fitting

“For the markdown file let’s just spit out all the output from the random effects model and the iccest command so we can show it to Grace. Eventually we can put the ICC in the table with the other repeatability measures you have already tabled but I don’t think we need it for the meeting. I also added a model to test whether SS influences reliability. The reference level is ‘10’ now, which is what we want, and you can see that SS is highly associated with VAD, meaning it affects reliability.” - BB 9/4/2019

## Linear mixed model fit by REML ['lmerMod']
## Formula: v_area_density ~ (1 | ptid)
##    Data: data_octa_all_intra
## 
## REML criterion at convergence: -1099.1
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.4075 -0.3684  0.0477  0.3807  2.6702 
## 
## Random effects:
##  Groups   Name        Variance  Std.Dev.
##  ptid     (Intercept) 0.0040672 0.06377 
##  Residual             0.0008736 0.02956 
## Number of obs: 320, groups:  ptid, 84
## 
## Fixed effects:
##             Estimate Std. Error t value
## (Intercept) 0.312238   0.007187   43.44

References

1. Venugopal JP, Rao HL, Weinreb RN, et al. Repeatability and comparability of peripapillary vessel density measurements of high-density and non-high-density optical coherence tomography angiography scans in normal and glaucoma eyes. The British Journal of Ophthalmology 2019;103:949–954.

2. Venugopal JP, Rao HL, Weinreb RN, et al. Repeatability of vessel density measurements of optical coherence tomography angiography in normal and glaucoma eyes. The British Journal of Ophthalmology 2018;102:352–357.

3. Bland JM, Altman DG. Statistics Notes: Measurement error. BMJ : British Medical Journal 1996;313:744.